Skip to content

feat(ci): reconcile advisory PR trust policy from current evidence - #383

Merged
dodeja merged 4 commits into
cursor/land-agent-trust-main-c4b3from
cursor/api-risk-gates-8528
Oct 17, 2026
Merged

dodeja merged 4 commits into
cursor/land-agent-trust-main-c4b3from
cursor/api-risk-gates-8528

Conversation

@dodeja

@dodeja dodeja commented Sep 27, 2026 •

Copy link
Copy Markdown
Member

Why

The original policy consumed racing labels, ran approval logic from PR-editable review workflows, and could mistake incomplete verification for success. This change evaluates fresh GitHub evidence in one default-branch controller.

What changed

  • Validate complete rename-aware paths, current-head human writer reviews, review revocation, and actual CI workflow/run/attempt/job evidence. Order exact attempt metadata by its start time and block any active matching attempt. Labels are outputs and cannot grant approval.
  • Replace the split classifier, gate, and eligibility reporter with one advisory controller. Pin its Actions and execute no PR code with write credentials.
  • Add policy and configuration regressions to required CI, a read-only installation checker, and a real GitHub trial guide.

Scope

This PR is explicitly advisory. It contains no App publisher, merge actuator, or low-risk human-review exemption. Native review protection must be installed separately. PR #384 provides its verifier foundation and must land first.

Tradeoffs

An Actions check name is not a trusted authorization identity. Keep auto-merge disabled and human CODEOWNERS review for every path until a separately reviewed publisher and merge-time verification trial establish a safe exemption.

Blast Radius

The workflow changes risk labels and Actions summaries only. Missing, stale, incomplete, conflicting, skipped, or neutral evidence cannot produce an automated-merge authorization. Hourly and review signals provide eventual refresh rather than immediate revocation guarantees.

Verification

  • Full npm run agent-verify, 40 focused policy/configuration regressions, actionlint, format/lint, and git diff --check passed locally.
  • A read-only replay with GitHub Script's pinned API client verified PR feat(ci): verify agent trust and preserve generated API contracts #384's actual current CI and correctly required human approval. Previous defects have red/green regression evidence.
  • The live configuration checker reports missing native protection on main. The installation guide and GitHub trial define the remaining activation proof.

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with a non-blocking issue in scheduled advisory refreshes.

Fix All in CodexFindings

  1. P2 One PR stops other refreshes ▶
Fix with agent prompt
### Issue 1
scripts/ci/reconcile-trust.mjs:344-345
Scheduled and untargeted manual runs stop updating all remaining PRs when one `reconcilePull` call throws. For example, two CI attempts with matching start times always throw during collection. That PR can therefore prevent later PRs from receiving fresh labels and summaries on every hourly run.

Catch errors per PR, continue refreshing the others, and report the failures afterward. This affects advisory updates only; stale labels do not grant merge permission.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
Summary

Adds one default-branch controller that reads current PR files, reviews, permissions, and CI evidence. Its labels and summaries remain advisory; it never authorizes auto-merge.

  • Checks both sides of renames and requires current-head human writer approval for medium and high risk.
  • Adds policy tests to required verification, catch-all CODEOWNERS coverage, and installation and trial guides.
  • One non-blocking issue remains: a failed PR refresh stops the scheduled refresh of unrelated PRs.
  • The eight earlier, unnumbered threads were rechecked. Their reported problems are addressed by the advisory design, rename checks, test wiring, and pinned actions.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Events[PR updates, CI completion, review signal, schedule or manual run] --> Controller[Default-branch controller]
  Controller --> Evidence[Read current files, reviews, permissions and CI attempts]
  Evidence --> Evaluate[Evaluate risk, approval and verification]
  Evaluate --> Reread[Read evidence again]
  Reread --> Output[Advisory labels and summary]
  Output --> NoMerge[Auto-merge always ineligible]
  Native[Separately installed native human review] --> Merge[GitHub merge protection]
Loading

Reviews (2) · Last reviewed commit: "Merge branch 'codex/trust-verifier-fixes..."

Current readiness verification keeps the original gates. Local checks and tests pass, but authenticated MCP preview evaluation is blocked by a backend HTTP 401 for the shared CI passthrough credential. The transport handshake alone does not prove tool access. Review fixes and rebases preserve this PR's base topology. No merge or deploy has been performed.

@mintlify

mintlify Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
terminal49 🟢 Ready View Preview Sep 27, 2026, 12:50 AM

@vercel

vercel Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
api Ready Ready Preview Oct 17, 2026 7:09am UTC

Request Review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T07:12:02.006851Z 0175bd7 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a05d286bdd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/risk-gate.yml Outdated
Comment thread .github/workflows/pr-risk-classifier.yml Outdated
Comment thread docs/risk-gates.md Outdated
Comment thread scripts/ci/classify-pr-risk.mjs
Comment thread .github/workflows/pr-risk-classifier.yml Outdated
Comment thread .github/workflows/risk-gate.yml Outdated
Comment thread .github/workflows/risk-gate.yml Outdated
Comment thread .github/workflows/pr-risk-classifier.yml Outdated
Comment thread .github/workflows/auto-merge-eligibility.yml Outdated
Comment thread .github/workflows/shadow-risk.yml Outdated
Comment thread package.json Outdated
Comment thread .github/workflows/pr-risk-classifier.yml Outdated
@dodeja dodeja changed the title feat: add path-based pull request risk gates feat(ci): reconcile advisory PR trust policy from current evidence Oct 6, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b89ef6f51e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ci/trust-policy.mjs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 98e521cee5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread skills/agent-trust/references/risk-gates-trial.md

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e043335934

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/shadow-risk.yml Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 74657aad7d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ci/reconcile-trust.mjs Outdated
Comment thread skills/agent-trust/references/risk-gates.md

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7893037698

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ci/read-risk-config.mjs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bc5b47296c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ci/read-risk-config.mjs Outdated
Comment thread Terminal49-API.postman_collection.json Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0968d11844

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ci/read-risk-config.mjs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2c45c20c88

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ci/trust-policy.mjs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 40d9a4b4bc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/shadow-risk.yml Outdated
dodeja and others added 3 commits October 17, 2026 00:07
Add an advisory trust-policy controller that classifies changed paths by
risk, checks current-head human writer approvals and exact-attempt CI
evidence, and writes risk labels and summaries. It never authorizes a
merge; native branch protection stays the merge authority.

- trust-policy.yml, shadow-risk.yml and trust-review-signal.yml run
  trusted default-branch code only.
- scripts/ci holds the classifier, the shadow scorer, the read-only
  branch-protection checker and the reconciler, with regression tests.
- CODEOWNERS names owners for every high-risk path.
- npm test and agent-verify now run the policy tests.

Rebuilt from the reviewed history at 6b5f819 on top of the rebuilt
agent-trust base; the Postman bot commit is not carried over.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep the maintainer guides with the agent-trust skill and link them from
the paved path, so maintainers run the GitHub verification trial before
they change merge protection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bin/check-openapi-types.test.mjs was classified medium risk and had no
CODEOWNERS entry, unlike the other verifier tests. The new test derives
the verifier list from bin/, so a future verifier cannot miss either.

The docs/risk-gates*.md entries pointed at guides that now live under
skills/agent-trust/, which the existing prefix rules already cover.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dodeja
dodeja force-pushed the cursor/api-risk-gates-8528 branch from 6b5f819 to 0175bd7 Compare October 17, 2026 07:08
@dodeja
dodeja merged commit e909f5d into cursor/land-agent-trust-main-c4b3 Oct 17, 2026
2 of 3 checks passed
dodeja added a commit that referenced this pull request Oct 18, 2026
* feat(ci): add the agent trust verification gate

`npm run agent-verify` is the local completion gate. It runs the agent
trust guards, the Vite+ and TypeScript checks, every workspace build and
test, the OpenAPI generation drift check, and the offline MCP protocol
smoke.

- bin/check-agent-trust.mjs keeps skills/agent-trust/feature-map.json in
  sync with the MCP tools registered in server.ts, the SDK package
  exports, and the MCP and SDK docs navigation. It also checks the
  vite-plus toolchain pins, the ordered workspace build, and that no
  workspace has its own Vite config.
- bin/check-openapi-types.mjs regenerates the SDK types, the SDK filter
  reference, and the MCP filter schemas from docs/openapi.json in a
  temporary checkout and fails on any drift.
- The feature map covers main as it is now: 14 tools (whoami included)
  and the Gemini and Antigravity setup guides.
- The guard skips the TypeScript build-info file when the SDK compiles
  incrementally or as a composite project. Before this, that file
  failed the dist check.
- CI runs the guards and the drift check as a new step in the toolchain
  job. The existing check, test, and build steps stay as they are.

Rebuilt on current main from #384's branch (head 4241fb6). The
/containers/{id}/route OpenAPI path, its schemas, and its reference page
are left out. Main removed those docs in 2792425.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(sdk): compile consumers against the generated container contract

Type-checks small consumer programs against the generated OpenAPI types.
A container payload with type "container" compiles. An account
discriminant is rejected with TS2322.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: keep the list filter reference SEO metadata in its generator

#421 edited the title and description in the generated list-filters.mdx
but not in the script that generates it. The next regeneration (#418)
put the old metadata back. This change moves #421's metadata into
generate-filter-types.mjs and regenerates the page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: Auto-generate Postman collection from openapi.json [skip ci]

* feat(ci): reconcile advisory PR trust policy from current evidence (#383)

* feat(ci): reconcile advisory PR trust policy from current evidence

Add an advisory trust-policy controller that classifies changed paths by
risk, checks current-head human writer approvals and exact-attempt CI
evidence, and writes risk labels and summaries. It never authorizes a
merge; native branch protection stays the merge authority.

- trust-policy.yml, shadow-risk.yml and trust-review-signal.yml run
  trusted default-branch code only.
- scripts/ci holds the classifier, the shadow scorer, the read-only
  branch-protection checker and the reconciler, with regression tests.
- CODEOWNERS names owners for every high-risk path.
- npm test and agent-verify now run the policy tests.

Rebuilt from the reviewed history at 6b5f819 on top of the rebuilt
agent-trust base; the Postman bot commit is not carried over.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: add the trust-policy installation guide and GitHub trial

Keep the maintainer guides with the agent-trust skill and link them from
the paved path, so maintainers run the GitHub verification trial before
they change merge protection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ci): own every agent-trust verifier and drop moved guide paths

bin/check-openapi-types.test.mjs was classified medium risk and had no
CODEOWNERS entry, unlike the other verifier tests. The new test derives
the verifier list from bin/, so a future verifier cannot miss either.

The docs/risk-gates*.md entries pointed at guides that now live under
skills/agent-trust/, which the existing prefix rules already cover.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: Auto-generate Postman collection from openapi.json [skip ci]

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* chore: Auto-generate Postman collection from openapi.json [skip ci]

* chore: keep main's generated Postman collection

The generator assigns random request IDs and enum example values, so
each rerun rewrites about 640 lines of the collection from an unchanged
docs/openapi.json. This branch does not change the spec, so it carries
main's collection and the PR diff has no generated churn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ci): hold aliased MCP registerTool calls to the feature map

The registration inventory only counted calls whose receiver was the
identifier `server`, so `const alias = server; alias.registerTool(...)`
or a reassigned receiver registered a tool the feature map never saw.

Every `<receiver>.registerTool(...)` call in server.ts now counts as a
registration whatever the receiver is. Any other use of `registerTool`
(destructuring, `.bind`, a computed key held in a variable) fails with
the source line, so a registration cannot hide behind an unsupported
form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ci): skip node_modules when expanding workspace globs

The toolchain guard appended /package.json to each root workspace pattern
and globbed it as-is, so a recursive pattern such as sdks/** also matched
installed dependency manifests (sdks/typescript-sdk/node_modules/*/package.json)
and failed on their missing toolchain pins. npm ignores **/node_modules/**
when it maps workspaces; the guard now does the same.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ci): give each trust reconciliation a 60s deadline

The advisory sweep awaited every PR in sequence, so one stalled GitHub
request could hold the job until its 20-minute timeout and starve every
later PR. Each PR now runs under an AbortController deadline. When it
fires, the PR is reported as failed, its in-flight direct request is
aborted, any later label write for that PR is rejected before it is sent,
and the sweep moves on to the next PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ci): keep timed-out PRs out of the trust job summary

A PR that hit its deadline could still finish later (a stalled paginated
call is not aborted) and append its report to the shared job summary,
next to the failure that said it timed out. Each PR now writes into its
own buffer, and only a PR that finishes within its deadline flushes it to
the job summary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

This branch was successfully deployed

2 active (1 outdated) deployments
Preview — e2966523 Deployed Oct 17, 2026 by vercel[bot]
staging - docs — 6b5f819c Deployed Oct 14, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant